1. Field of the Invention
The present invention relates to an image pickup apparatus provided with a solid-state image pickup device, such as a complementary metal oxide semiconductor (CMOS) image sensor, a method of correcting image data captured by the image pickup apparatus, and a program executing the method.
2. Description of the Related Art
Solid-state image pickup devices, such as charge coupled devices (CCD) image sensors and CMOS image sensors, are configured such that microlenses for condensing light are layered on color filters under which pixel cells including photoelectric transducers are provided.
A problem called “pixel mixture” is unable to be disregarded in such an image pickup device as the pixels are increasingly reduced in size with the increasing density of the pixels in recent years. The pixel mixture is caused because the pitch of the pixels is decreased and light that is condensed by the microlenses on the image pickup devices and that passes through the color filters not only is incident on a pixel on which the light is to be condensed but also tends to leak into adjacent pixels.
FIGS. 18A and 18B illustrate how the pixel mixture occurs. In the examples shown in FIGS. 18A and 18B, the image pickup device is a CMOS image sensor 101 in which condenser lenses 104 composed of microlenses are layered on color filters 103 under which pixel cells 102 are provided. An aperture 105 is provided above the image sensor 101.
When the aperture 105 is not sufficiently narrow as in the example shown in FIG. 18A, light incident on the image sensor 101 through the opening of the aperture 105 is incident on not only one target pixel cell 102 but also adjacent pixel cells 102. Accordingly, the light passing through the color filters 103 other than the color filter 103 for the own pixel cell 102 is incident on each pixel cell 102. From the viewpoint of one pixel, part of the light that should be incident on the pixel leaks from the pixel into pixels around the pixel and part of the light that should be incident on the pixels around the pixel leaks into the pixel. Such a phenomenon is called the pixel mixture.
In contrast, when the aperture 105 is sufficiently narrow and the light incident in the image sensor 101 is incident on only one pixel cell 102 as in the example shown in FIG. 18B, the pixel mixture does not occur.
Since an occurrence of the pixel mixture degrades the separation between the adjacent pixels, an output image is reduced in resolution. In addition, in the case of a single-plate image pickup apparatus (camera system) having one solid-state image pickup device, since light passing through the color filters of different colors leaks into the pixels, the color of an image can become lighter and/or a geometrical pattern depending on the amount of color mixture can be produced in an even area of the image (an area where the content of the image is not varied or is slightly varied). In general, these problems suddenly cause adverse effects on the quality of output images as the pitch of the pixels is made narrower.
The leakage of light from a pixel into adjacent pixels around the pixel and the leakage of light from the adjacent pixels into the pixel are called the pixel mixture in this specification not only when the color of the color filter for the own pixel cell is different from the colors of the color filters for the adjacent pixel cells but also when the colors of the color filters for the adjacent pixel cells are the same as the color of the color filter for the own pixel cell.
Accordingly, in addition to the case where the color filters of different colors are layered on the pixel cells as in the single-plate color image pickup apparatus, also when the color filters for each image pickup device have the same color as in a three-plate color image pickup apparatus having one solid-state image pickup device for the color filter of each color, the state in which light incident on the image pickup apparatus is incident on not only one pixel cell but also adjacent pixel cells is called the pixel mixture.
In the above case, the pixel mixture can cause the resolution to be reduced and the pixels that should have the same color and brightness can be varied in contrast.
Technologies for correcting the pixel mixture in order to resolve the above problems include a technology disclosed in Japanese Unexamined Patent Application Publication No. 2007-142697. In a basic process of the correction technology disclosed in Japanese Unexamined Patent Application Publication No. 2007-142697, a value given by multiplying the difference between the value of a target pixel to be corrected, supplied from the solid-state image pickup device in an image pickup apparatus, and the value of an adjacent pixel by an appropriate correction factor (correction parameter) is added to the value of the target pixel. Provision of directionality (the difference in the direction and position between the target pixel and the adjacent pixel) to the correction factor and control by using an F-number are described in Japanese Unexamined Patent Application Publication No. 2007-142697.
In recent years, there is an increasing need for solid-state image pickup devices, such as the CCD image sensors and the CMOS image sensors, used in video cameras and digital still cameras to increase the number of pixels and to reduce in size. The increase in the number of pixels with the reduction in size causes the area of each pixel to be decreased, thus possibly lowering the pixel sensitivity.
In order to resolve such a problem, the proportion of photodiodes in the pixel structures of the solid-state image pickup devices has been increased as much as possible by devising the circuitry and the wiring layout and/or developing the manufacturing process to achieve both the increase in the number of pixels and the reduction in size. Each pixel structure is composed of the photodiode and a peripheral circuit including amplifier transistors and reset transistors.
However, the recent demand for the solid-state image pickup devices to increase the number of the pixels and to reduce in size is increasing and an improvement in the image quality at lower illumination also attracts attention. Accordingly, it may not be possible to provide the solid-state image pickup devices capable of achieving the pixel sensitivity that ensures a desirable image quality only by the above actions.
In such a situation, pixel sharing technologies receive widespread attention in order to overcome the above drawbacks of the solid-state image pickup devices. In the pixel sharing technologies, part of the circuit necessary for the pixel structure, for example, the amplifier transistors and/or the reset transistors, are shared between multiple pixels that are horizontally and/or vertically adjacent or close to each other to reduce the area of the circuit (including the wiring) per pixel and to increase the area of the photodiode by the amount corresponding to the reduction in the area of the circuit per pixel, thereby improving the pixel sensitivity.
For example, Japanese Patent Application No. 3838665 discloses a pixel sharing technology for sharing the amplifier transistors and the reset transistors necessary to read out pixel data from each pixel between two adjacent pixels. In this technology, the pixel data is read out from each pixel while the two adjacent pixels connected to the amplifier transistors and the reset transistors are sequentially selected at slightly different times to decrease the number of transistors per pixel and to increase the area of the photodiode by the amount corresponding to the decrease in the number of transistors per pixel in order to realize the improvement of the pixel sensitivity.
All the pixels are normally structured evenly in general solid-state image pickup devices that do not adopt the pixel sharing technology. In other words, the pixel structure of the general solid-state image pickup devices is common to the pixels at any positions on the screen. Accordingly, the photodiodes for all the pixels have the same peripheral environment in the semiconductor structure in such general solid-state image pickup devices. Consequently, the optical characteristics are basically common to all the pixels if the factor of any variation in the manufacturing is excluded.
In contrast, in a typical solid-state image pickup device adopting the “pixel sharing technology” disclosed in, for example, Japanese Patent Application No. 3838665, all the units have the same structure when the multiple pixels that share the circuit and that are adjacent or close to each other are considered as one unit while the pixels arranged at different positions in each unit have different peripheral environments in the semiconductor structure. Accordingly, the arrangement pattern of the multiple pixels sharing the circuit is repeated to produce the circuit layout in the solid-state image pickup device adopting the pixel sharing technology.
Specifically, when the multiple pixels sharing the circuit are considered as one unit, the multiple units are repetitively arranged horizontally and vertically in the solid-state image pickup devices adopting the pixel sharing technology. Since the photodiodes for the pixels arranged at the same position in the arrangement pattern of the multiple pixels in the multiple units have the same peripheral environment in the semiconductor structure, the pixels have the same optical characteristics.
However, the pixels arranged at different positions in the arrangement pattern of the multiple pixels sharing the circuit in the multiple units, that is, the pixels that are adjacent or close to each other in each of the multiple units have different circuitries and/or layouts. Accordingly, the photodiodes for the pixels have different peripheral environments in the semiconductor structure, thus providing different pixel characteristics.
Consequently, even if an image of a fully even subject is captured by the solid-state image pickup device adopting the “pixel sharing technology”, different pixel values are output from adjacent pixels in each unit and, thus, there is a problem in that the quality of the final output image may be greatly lowered.
In related art, in order to resolve the above problems involved in the pixel sharing, many technologies for devising the pixel structure of the solid-state image pickup device are disclosed in, for example, Japanese Unexamined Patent Application Publication No. 2004-172950, Japanese Unexamined Patent Application Publication No. 2005-110104, Japanese Unexamined Patent Application Publication No. 2006-73733, and Japanese Unexamined Patent Application Publication No. 2006-157953. In such technologies, for example, (A) the pixel layout is devised to reduce the optical unevenness as much as possible or (B) the pixel sharing is performed so as not to adversely affect the output images if the optical unevenness occurs.